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The ability to recognize equivalent algebraic expressions quickly and confidently is 
important for doing mathematics in an intelligent partnership with computer algebra. It 
is also a key aspect of algebraic expectation, the algebraic skill that parallels numeric 
estimation. The progress in this ability of 50 students learning mathematics with CAS 
over two school years was monitored using a novel instrument called the Algebraic 
Expectation Quiz. Students began with very low facility and confidence and made 
moderate progress, confirming the importance of addressing this obstacle to using CAS 
explicitly in the curriculum. We offer conclusions about test items, students’ strengths 
and weaknesses and suggest possible teaching actions. The results demonstrate that 
recognizing equivalence, even in simple cases, is a significant obstacle for students. 

INTRODUCTION 

The purpose of this paper is to report on students’ ability to recognize the equivalence of 
algebraic expressions and their confidence in their judgments, obtained from a novel 
instrument called the Algebraic Expectation Quiz. We report the results of using this 
instrument with 50 students over 21 months, during their final two years of secondary 
schooling learning mathematics with CAS, drawing conclusions about test items, 
students’ strengths and weaknesses and finally suggesting possible teaching actions. The 
results demonstrate that recognizing equivalence is a more significant obstacle for 
students than we and their teachers had expected and that students lack confidence. 

The work in this paper is motivated by the need to develop and monitor students’ 
algebraic thinking when computer algebra systems (CAS) are available for doing, 
teaching and learning mathematics. It has been carried out as part of a project (see CAS- 
CAT Research Project website) where students at three secondary schools studied a new 
tertiary-preparation mathematics subject for which CAS calculators (graphics calculators 
that additionally have a symbolic manipulation facility) were available at all times, 
including for examinations. Students learning and doing mathematics using CAS will not 
need the same mastery of algebraic routines as do other students. They will be able to use 
CAS where other students need a by-hand technique. However to use CAS effectively 
and to attain other educational goals, students will still require conceptual understanding 
and technical facility with algebra, the exact nature of which is a topic of debate and 
concern in the literature (see, for example, Drijvers (2000); Herget, Heugl, Kutzler & 
Lehmann (2000); Pierce (2002)). Artigue (2001) suggests that there is a complex 
dialectic between conceptual and technical work in algebra. Passing technical work over 
to CAS means that it still needs attention, although possibly different attention, in the 
curriculum. Controlled use of technology (such as CAS) almost certainly requires 
specific competencies that are not covered by the standard curriculum. 
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Equivalent forms and Algebraic Expectation 

The ability to recognise equivalent forms of algebraic expressions, which is the focus of 
this paper, has been identified as a central part of working with CAS, and one that is 
likely to take on new importance in future curricula. Many researchers, for example Heid 
(1989), Lagrange (1999) and Drijvers (2000), comment on the importance of this ability 
for a CAS user. Firstly, when students are entering expressions with complex syntax into 
a CAS they need to be able to recognise whether the expression entered is equivalent to 
the one they are working from on paper. For example, in order to enter into a CAS the 
expression sin 2x, which requires no brackets in by-hand mathematics, students need to 
appreciate that brackets are required in (most) CAS and then that sin (2x) is not the same 
as (sin 2)x. To enter (a + p)/q (possibly written with a vinculum instead of brackets), 
students must appreciate that this is not the same as a + p/q. Secondly, students need to 
be able to convert output into a standard form since CAS does not always present results 
in the manner which is conventional in a given educational setting. The roots of the 

quadratic equation x 2 -ltix-\6 = 0, for example may be given as V41 + 5 and -( V41 - 5), 
whereas teachers in our local schools would almost invariably give these as 5 + V41 and 

5 - V41 . Thirdly, students need knowledge of equivalent expressions to be able to deal 
with the automatic simplification feature of CAS. This feature means that an input is 
often processed to an equivalent form immediately on entry. When an expression such as 
2 f-g + 3f- g is entered into a CAS such as the TI-89, the input is automatically 
simplified and appears as 5/- 2 g, the input 1 2x/6x appears as 2, (b + a) 1 is reordered 
alphabetically and appears as (a + b) 2 and 6 + (4a + 2b) 12 appears as 2a + b + 6. 
Lagrange (1999) reported that students reflected deeply on events like these, when 
teachers wanted them to consider deeper phenomena. Effective CAS users need to be 
able to quickly recognize that the inputs have been made correctly, even though the form 
appearing on the screen is different. Fourthly, on-going monitoring of the progress of a 
calculation requires a general alertness to the appropriateness of intermediate results and 
quick, confident decisions. At all stages of their work, CAS users need to recognise 
simple equivalences quickly and confidently, so that they are not derailed by the trivial 
but are alerted to the significant unexpected. Guin and Trouche (1999) noted that a 
surprising CAS result did not necessarily induce questioning in students who could not 
compare expressions. 

We see the ability to recognize equivalent expressions as one indicator of a more general 
ability of algebraic expectation (Pierce, 2002; Pierce & Stacey, 2002). An important 
component of the algebra needed when CAS is used is ‘symbol sense’, rather analogous 
to the ‘number sense’ that is needed for doing arithmetic with a four function calculator. 
Arcarvi (1994) outlines what ‘symbol sense’ might be and how it is involved in all 
aspects of solving problems with algebra, including formulating problems algebraically, 
technical work within the mathematical world and interpreting algebraic answers in a real 
context. This describes the understanding of algebra required for working in partnership 
with technology. Within this overarching symbol sense, Pierce and Stacey (2002) have 
identified algebraic expectation as the algebraic parallel to numerical estimation. It is the 
thinking process that takes place when a mathematician considers the nature of the 
symbolic result expected as the outcome of some algebraic process. Its essence is making 
a quick assessment of the expected characteristics of algebraic results. Pierce and Stacey 
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(2002) define and analyse algebraic expectation in terms of several different elements: 
knowing conventions and basic properties of operations, and being able to identify the 
structure and key features of algebraic expressions. Algebraic expectation is the general 
ability needed to monitor the succession of expressions appearing on a CAS screen, 
making on-going rough checks for mathematical sense. 

Because of the centrality of the ability to quickly recognize algebraic equivalence to the 
more general ability of algebraic expectation, we call the instrument described below the 
Algebraic Expectation Quiz. However, adequate monitoring of algebraic expectation 
requires other assessments, so in the CAS-CAT project (CAS-CAT Research Project) the 
Algebraic Expectation Quiz was supplemented by two tests that are not reported here: a 
6-item Constructed Response Test and a separate test of both multiple choice and 
constructed response items assessing the ability to link representations. 

THE ALGEBRAIC EXPECTATION QUIZ 
Design of instrument 

Outside the CAS experience, one of the daily experiences students have of needing to 
recognise equivalence of algebraic expressions is in checking their work from the back of 
a textbook. Very often, a student’s answer is not in the same form as the answer in the 
book and so the student must decide whether their answer is correct or not. This situation 
was used to supply a realistic context for the Algebraic Expectation Quiz. Students were 
told that they would be presented with a series of slides, each showing two expressions. 
One expression was the answer given by a mythical student (Julie) to a problem and the 
other was the textbook answer. For example, the first item (item 1 of Table 1) presents on 
one slide x+y (Julie’s answer) and y-s-x (the textbook answer). The students’ task was to 
decide if Julie was definitely wrong, probably wrong, probably right or definitely right. 
Students unable to decide could choose the option ‘no idea’. This mechanism enables a 
measure of confidence in the decision to be obtained simultaneously with accuracy. To 
reduce confusion of right/wrong and correct/incorrect in reporting results, we will refer to 
the items where Julie’s answer matched the textbook answer as true items and the other 
items as false. 

The Algebraic Expectation Quiz tries to capture the essence of making quick, real-time 
decisions, mirroring the way in which algebraic expectation provides constant and almost 
unconscious on-going monitoring of algebraic results. Hence it was important to present 
items with a restricted time to respond. To do this, the test was designed to be 
administered using Microsoft PowerPoint with a fixed time of 10 seconds for each item 
(time based on prior trialing by Pierce (2002)). Because of the need for unbroken 
concentration, the test time cannot exceed about 5 minutes, so that approximately 25 
items can be used. 

Items were selected based on previous research on learning algebra or using CAS (see for 
example Drijvers, 2000; Kieran,1992; MacGregor and Stacey, 1997)), Pierce’s quiz 
(2002) and the researchers’ experience of working with students at this level. To confirm 
appropriateness and validity, items were reviewed by experienced mathematics teachers, 
who were asked whether they expected that students in Years 11 and 12 could quickly 
establish equivalence (or otherwise) of the expressions without any written steps. 
Whether or not a student can be said to have “good algebraic expectation” depends on 
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their age and stage, so this heavily influences the creation of items. With few exceptions, 
we chose items from curriculum topics in Year 10 and below, using the four basic 
operations and square root. Informative items will canvas topics that a teacher assumes 
students already confidently know as well as what they need to know for their coming 
studies. Links between the items and skills required for using CAS can be seen by 
comparing the items with the examples used in the Introduction to this paper. 

Table 1 lists 22 items, ordered according to item facility at the first test time. All items 
involve knowledge of conventions and basic properties of operations (for example that 
addition is commutative, the square root of a sum is not the sum of square roots), but this 
is not always the source of challenge for an item. Analysis of the first results of 
administering the Algebraic Expectation Quiz (Ball, Pierce & Stacey, 2001) showed that 
the most effective items required at most two intermediate steps which were simple 
enough to be carried out mentally. For example, Item 12 requires canceling the negatives 
and splitting the fraction. More complicated items were discarded. 

Data collection 

Along with other assessments, the Algebraic Expectation Quiz was administered to 
students in three project schools four times during the 2001 and 2002 school years to 
monitor their developing algebraic knowledge and skills. The complete cohort contained 
students learning with CAS and without it. However, the results in this paper are from the 
fifty students who completed the test in both February 2001 (beginning of Year 11) and 
September 2002 (just before the end of Year 12 and their final external examinations) and 
who undertook the new mathematics subject that permitted CAS. The results presented in 
this paper are for the 22 items common to the February 2001 (Feb 01) and September 
2002 (Sep 02) administrations of the quiz. Information about the three project schools 
indicates that the 50 students may have academic ability better than the state average. 

The quiz was administered by two of the present authors and one other researcher. They 
began by outlining the purpose of the quiz, explaining that each slide would only be 
shown for ten seconds and that there would not be enough time to do any written working 
but that students were expected to make a quick judgment based on their mathematical 
experience. Two sample items were demonstrated before the timed sequence began and 
they were used to stress that the students needed to judge the algebraic equivalence of the 
expressions rather than nicety of form. It was also explained that the intention was not to 
look for special cases, but to consider general expressions. For example, x+y is indeed 
equal to y+x when x = ±y, but not in general, so Julie’s answer in Figure 1 is wrong (i.e. 
Item 1 is a false item). The Algebraic Expectation Quiz took less than 5 minutes followed 
by other assessments not reported here. Students completed the Algebraic Expectation 
Quiz without use of pen and paper or technology except to record answers. 

RESULTS 

For each item Table 1 reports (i) the facility (% of students answering correctly i.e. 
definitely right or probably right on true items, and definitely wrong or probably wrong 
on false items) and (ii) percentage of students with well-placed certainty (both certain and 
correct i.e. answering definitely wrong for a false item or definitely right for a true item) 
and (iii) misplaced certainty (certain and incorrect i.e. definitely wrong for true or 
definitely right for false) in Feb 01 and Sep 02. 
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Table 1: Algebraic Expectation Quiz results Feb 2001 and Sept 2002 (N=50) 

The facilities in Table 1 show that the items covered a wide range of difficulty. For 
example, Item 6 (the comparison of 5m with m 5 ) is a false item, and in Feb 01, 92% of 
students answered correctly definitely wrong or probably wrong. By Sep 02, this had 
increased to 100% suggesting that students had good algebraic expectation for items of 
this type for the duration of the study. The table also indicates that most of the students 
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were certain of their correct result (78% in Feb 01, 94% in Sep 02). If an erroneous 
answer of this type was obtained from CAS work, we would expect students to identify 
it. Notational items (items 6 and 24) were well done as was linear factoring (item 22) and 
collecting like terms (item 13). Simple reorganizations such as the slightly unusual 
presentation of ( a+b ) 2 (item 23) were unexpectedly difficult although there was a 
significant improvement in both facility (Wilcoxon signed ranks test comparing median 
item scores for Feb 01 and Sep 02: W=228.5, p=0.000) and well-placed certainty 
(W=226.5, p=0.000) in Sep 02. Almost all fraction items (for example, items 12 and 21) 
had very low facility on both occasions. The relatively high percentage of students with 
misplaced certainty shows that many students remain unaware of their errors. On 
average, however, the facility of items improved by 20% over the two years, as did the 
well-placed certainty, whilst happily the misplaced certainty decreased (W=21, p=0.001). 

Figure 1 shows that certainty (total of well-placed and misplaced certainty) and item 
facility are moderately related (correlation Feb 01=0.60, Sep 02 =0.76). The scattergrams 
show that items with highest facility in Sep 02 generally had high certainty, whereas in 
Feb 01 there was lower correlation. There are many interesting individual items. For 
example, in Sep 02 confidence for item 15 remains low, despite high facility. On this 
item, in Feb 01 students displayed misplaced certainty; in Sep 02 more are correct but 
they are unsure of their response. Items 4, 1 1 and 20 have low success and high certainty. 
In Feb 01, about half of the students showed misplaced certainty, and still about one third 
in Sep 02. These items correspond to persistent misconceptions (see MacGregor et al, 
1997, for example). We conclude that many students are unlikely to identify errors of this 
nature in their work. In contrast, the three most difficult items (12, 16 and 21) have low 
facility, but moderate certainty. By Sep 02, only 13% showed misplaced certainty on 
these items. Although they were not sure of the correct answers, most students had at 
least become aware that these items represented areas where care is needed. 
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Figure 1: Scattergrams of Certainty versus Item Facility for Feb 01 and Sep 02 
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DISCUSSION 



This discussion of the usefulness of the Algebraic Expectation Quiz as an instrument, 
information about students’ algebraic expectation, and implications for teaching, is based 
on the results, together with feedback from the researchers who administered the quiz. 

The Algebraic Expectation Quiz format was very successful. Students understood the 
scenario of comparing student and textbook answers and enjoyed the fun, fast Powerpoint 
presentation. The opportunity to indicate certainty reduced pressure on students and it 
provided deeper insight into their algebraic thinking. Since the quiz is quick to administer 
and easy to assess, the format is likely to be useful. A disadvantage is that the quiz format 
requires continuous concentration and therefore the number of items is limited. 

The quiz results provide a partial but useful indicator of students’ algebraic expectation, 
specifically their ability to recognize equivalent algebraic form. Overall, the results 
demonstrated both low facility and low levels of confidence. Improvement over two 
years was moderate, given that algebra becomes more important in the curriculum during 
years 11 and 12. Improvement in well-placed certainty was also moderate, although 
accompanied by a pleasing reduction in misplaced certainty. It is likely that students were 
increasingly aware of areas where they needed to be careful. The identification of areas 
of misplaced certainty (such as items 21, 12, 20, 3) provides information to teachers 
about firmly entrenched student misconceptions. Lack of certainty may also lead to 
excessive reliance on the machine: we have sometimes observed students who insist on 
copying machine output absolutely precisely, including factors of 1 or ln(e), for example. 

Overall, the low facility and low confidence demonstrate that students may benefit from 
the support of CAS in more situations than teachers expect. At the same time, it is likely 
that they will find CAS harder to use than might be expected, because of the difficulties 
outlined above related to identifying equivalent algebraic forms. This is especially so, 
given that participating schools were of above average academic attainment. 

As noted, to develop controlled use of technology, it is likely that the standard curriculum 
will need to change. The results show that giving more emphasis to algebraic expectation 
and considering a wider range of algebraic forms is a vital direction for change. Targeting 
items to known CAS peculiarities may be useful e.g. changing the notation of the hand- 
written vinculum for division to the one line calculator format using a division sign or 
negative power. Pierce, who studied tertiary students learning with CAS extensively and 
documented their progress (Pierce, 2002) suggests that teachers discussing expressions 
may routinely ask about structure and key features. This can take little time, and become 
a routine for teacher and students, which develops algebraic expectation. Lagrange 
(1999) and Guin and Trouche (1999) have also been concerned to develop strategies. 

Managing the tensions created by differences in algebraic form between by-hand and by- 
CAS algebra was always a concern for teachers working in the CAS-CAT project. In an 
interview at the end of the second year of teaching with CAS Lucy, one teacher in the 
CAS-CAT project, commented that her students were increasingly better able to deal 
with differences in form and she identified a teaching strategy developed to help them: 

“I’m finding increasingly that when the kids have done it by-hand and then they use the CAS, 

they’re actually better prepared for the differences in form. And I notice one other thing. 
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When I’m doing symbolic procedures more often at the end now, I’ll write three or four 
answers and then I’ll ask the kids, ‘Which one did the calculator give you?’ . . . Sometimes the 
way that it will express something that has surds or has fractions is not consistent [with by- 
hand conventions] so . . . you have to be able to be confident to say that’s just the same thing.” 

As teachers like Lucy experiment with teaching with CAS, experience of how to develop 
better algebraic expectation will grow. Algebraic knowledge is required to identify 
equivalent forms (or non-equivalent forms if errors are made) quickly, with well placed 
certainty, for on-going monitoring. The data presented in this paper has demonstrated that 
students may have greater needs in this area than have been appreciated. 
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